Findings from a Multi - Center Study of Youth Outcomes in Private Residential Treatment

This paper presents the results from the first phase of a longitudinal, multi-center study of outcomes in private residential treatment. It is the first known large-scale attempt at a systematic exploration of client characteristics, treatment outcomes, and discharge predictors in private residential treatment. The sample of nearly 1000 adolescents, from nine private residential programs, was about equally likely to be male or female, from middle or upper socioeconomic backgrounds and predominately white. Ninety-five percent had prior treatment and 85% were treated for multiple presenting problems, the most common of which were disruptive behavior, mood, and substance abuse problems. Parents and adolescents reported significant improvement during treatment on adolescent communication, family relationships, and compliance. Analyses of variance indicated that both adolescents and parents reported a significant reduction in problems from admission to discharge, on each aggregate measure psycho-social functioning (Total Problems Scores, Internalizing Scales, and Externalizing Scales of the Child Behavior CheckList, CBCL, and Youth Self-Report, YSR) and every syndrome (16 YSR and CBCL Syndrome scales). Only two out of 22 treatment and non-treatment-related variables (Grade Point Average and Mood Disorder) interacted with outcomes. Furthermore, in stepwise regression analyses, testing a wide array of treatment and non-treatment variables, only a handful of variables predicted discharge functioning. Taken together, the analyses suggested that adolescent problems improve significantly during private residential treatment and that, with only a few exceptions, discharge functioning and in-treatment change are relatively similar, regardless of adolescent background, history, problems, and treatment factors. Implications and research recommendations are presented. INTRODUCTION There is a dearth of published outcome research conducted in residential treatment and the existing research is fraught with problems (Curry, 1991; Epstein, 2004). One issue with this body of literature pertains to the samples, which were drawn primarily from public residential treatment programs (Curtis, Alexander, & Longhofer, 2001; Hair, 2005). Public residential treatment clients are typically referred thru public avenues (juvenile justice system, child protection agencies, or public mental health systems) (Curtis, et. al., 2001; Epstein, 2004; Hair, Residential Outcomes., p. 2 2005) and funded with public money. Furthermore, they are predominantly males, who are disproportionately from ethic minority backgrounds. In private residential treatment, adolescents are typically placed by their parents, who pay for treatment. Though no client demographic and background information are available for private residential treatment programs, informal observation across a variety of programs suggests that clients are equally likely to be male or female, are predominately white, and come from upper middle class or upper class socioeconomic backgrounds. Based on the foregoing, there is a possibility that public residential treatment and private residential treatment are different services, utilized by different client populations. Nonetheless, literature reviews conceptualize private and public residential treatment research as one corpus of literature. Further complicating the issue, private residential treatment programs are so heterogeneous in terms of their treatment philosophy and services that in the relatively rare case of published research based on samples from private residential treatment, there is uncertainty about the degree to which the findings generalize to other private residential treatment programs (Curtis et al., 2001; Woodbury, 1999). Another set of concerns about the residential treatment research corpus pertains to methodological flaws. Reviewers criticize this research for its poor samples, retrospective designs, unstandardized measures, and unsophisticated statistical analyses (Curry, 1991; Curtis et al., 2001; Epstein, 2004, Hair, 2005). The majority of studies use only one informant, even though the value of multiple informants has been established (Rend, 2005). Many studies use self-styled measures that lack normative data and psychometric rigor (Hair, 2005). There is lack of consensus on the timing and method of outcome measurement, making it difficult to integrate findings across studies. Sample sizes tend to be very small. Relatively few studies use advanced statistics that control for error or explore the impact of moderator and predictor variables. Reviewers have concluded that the effectiveness of residential care is largely unresolved in the research because of these and other methodological shortcomings. Despite these limitations several conclusions have been drawn pertaining to this body of literature. Most adolescents improve during residential treatment. Though reported outcomes vary widely, ranging from about 25 % to 80%, reviews suggest that 60%-80% of adolescents improve during residential treatment (Curry, 1991; Curtis et al., 2001; Epstein, 2004; Hair, 2005; Wells, 1991). The following factors have been shown to predict outcome: age, intelligence, degree of pathology, stability of the discharge placement, aftercare participation, and the absence of externalizing behaviors (Connor, Miller, Cunninghan, & Melloni, 2002; Epstein, 2004; Gorske, Srebalus, Walls, 2003; Wells, 1991) Specifically, research suggests that adolescent females with high IQ, less severe dysfunction, acute and late onset, better academic ability, absence of learning disorders, low levels of behavioral problems, and high levels of internalizing problems tend to have positive outcomes. Some researchers have concluded that residential treatment is best for higher functioning, less vulnerable youth (Connor et al., 2002). One recent study that sampled from 17 public residential treatment programs, found that age and race predicted outcomes (Lyons, McCulloch, & Romansky, 2006), with mid-aged adolescents and African American adolescents having relatively worse outcomes. It bears repeating that the degree to which the findings within this corpus of research apply to private residential treatment programs is largely unknown. This study attempted to add to the residential treatment literature by using a multi-center design, with repeated and standardized measures, prospective data, a large sample, and two informant groups. The questions were: 1) How do the two groups of informants (adolescents and parents) compare on their report of outcomes? Residential Outcomes., p. 3 2) What are the characteristics of adolescents treated in private residential care? 3) To what degree do adolescents reportedly change during the course of treatment? 3a) How does reported adolescent functioning vary across the selected treatment outcomes (e.g., aggressive behavior, anxious/depressed symptoms, attention problems, aggressive behavior, communication quality, family relationships)? 3b) Do youth outcomes vary according to non-treatment factors (e.g., age, gender, prior treatment history, legal record, type of presenting problems, and number of presenting problems)? 3c) Do youth outcomes vary according to treatment factors (e.g., adolescent’s response to placement at admission, length of stay, discharge status, satisfaction with treatment)? 4) What factors (treatment and non-treatment) predict adolescent functioning at the point of discharge? METHOD Participants. The sample consisted of 993 adolescents, admitted to one of 9 programs located in the Eastern and Western United States, between August 2003 and August 2005, who, along with their parents or guardians (hereafter referred to as “parents”) agreed to participate in the study and who completed measures at admission and/or discharge. The Western Institutional Review Board approved consent/assent forms and issued Certificates of Approval for the study. The contribution of each of the 9 residential programs to the sample was relatively equal and ranged from 9% to 16%. This sample consisted of a mean of 55% (range 37-75%) of the adolescents admitted to the residential programs during the time period. Demographic information (i.e., gender, age) from admission data provided by the residential programs indicated the sample was roughly representative of students enrolled in the programs during the same time period. Description of the residential programs. The 9 participating programs were private, out-of-home, licensed, therapeutic placements for adolescents and are member-programs of the National Association of Therapeutic Schools and Programs (NATSAP): Academy at Swift River, Aspen Ranch, Copper Canyon Academy, Mount Bachelor Academy, Stone Mountain School, Pine Ridge Academy, SunHawk Academy, Turnabout Ranch, and Youth Care, Inc. Residential treatment is a complex service that utilizes various approaches to the treatment of serious emotional and behavioral problems. Most adolescents are placed in private residential treatment by their parent(s): the juvenile justice and child welfare systems typically do not refer to these programs. Programs have on-site schools or academic programs and multi-disciplinary treatment teams. All programs provide group, individual, and family services, but the amount and type of each varies widely among the programs. During treatment, adolescents progress through “levels” associated with increasing privileges (e.g., possession of personal items, home visits) and responsibilities (e.g., peer mentoring, community leadership). Days are highly structured, with most time spent in school, community meetings, treatment groups, recreation, or counseling. Professional program staff includes social workers, psychologists, substance abuse counselors, marriage and family Residential Outcomes., p. 4 therapists, counselors, teachers, nurses, and psychiatrists. The participating residential programs vary widely in terms of size (15-bed to 120-bed), location (Massachusetts, Utah, Arizona, Oregon, North Carolina), treatment philosophy (therapeutic boarding school or residential treatment, the latter of which is more clinically focused and designed for more severely impaired adolescents), and the range of interventions provided (e.g., equine therapy, neurofeedback, adventure therapy, partial community placements). The diversity of the participating programs is reflective of the diversity within the private residential treatment industry. Design and Measures. The data (N=993) formed three subsets, determined by the availability of admission and discharge data. The data subsets are 1) admission, 2) treatment outcome (admission and discharge), and 3) discharge. A single-group design was used for all subsets of data. A pretestpost-test design was used for the treatment outcomes data set (N of adolescents =403, N of parents=211) which was derived from parents and adolescents who completed measures at admission to and, subsequently, discharge from a program. Its purpose was to examine change in functioning during treatment. The admission data set (N of adolescents=754, N of parents = 635) was derived from parents and students who completed measures at admission. This data subset provided information on the characteristics of adolescents treated at private residential programs. The discharge subset of data (N of adolescents=616, N of parents = 404) was derived from parents and students who completed the measures at discharge and was used to identify predictors of discharge functioning. For all data subsets the primary measures were the Child Behavior Check List (CBCL) and the Youth Self Report (YSR) (Achenbach, 2001), two widely used measures of adaptive and maladaptive psycho-social functioning. The reliability and validity of the CBCL and YSR Syndrome Scores, Internalizing and Externalizing Scores, and Total Problem Score have been demonstrated (e.g., Achenbach, 2001; Bérubé, & Achenbach, 2006). The CBCL is a parentreport measure of adolescent functioning that consists of 113 items. The YSR is a self-report measure that consists of 112 items. The measures have the same item format and scales, making them highly compatible. Items are rated on a three-point scale and are primarily objective or behaviorally anchored (e.g., “cries a lot”, “gets teased”, "fidgets”, “truant”). For the purposes of this study the following YSR and CBCL scales were used: Syndrome Scales (Anxious/Depressed, Withdrawn/Depressed, Somatic Complaints, Thought Problems, Attention Problems, RuleBreaking Behavior, Aggressive Behavior), Internalizing (problems that are mainly within the self), Externalizing (problems that mainly involve conflict with other people and their expectations for the child), and Total Problems Score (the sum of all the problems reported on the measure). High scores on a scale indicate clinical deviations from the norm and the presence numerous problems. Each raw scale score can be converted into a T score, percentile rank, and range (Normal, Borderline Clinical, and Clinical). Scores are truncated on the Syndrome Scales at the 50 percentile. The Normal range spans the 50 to the 92 percentiles on the Syndrome Scales and 24 to the 84 percentile on the Internalizing, Externalizing, and Total Problems Scales. The Borderline Clinical range spans the 93 to 96 percentile on the Syndrome Scales and 84 to 91 percentile on the Internalizing, Externalizing, and Total Problems Scales. The Clinical range on the Syndrome Scales spans the 97 to 100 percentile and on the Internalizing, Externalizing, and Total Problems Scales spans the 92 to 100 percentiles. As recommended in the manual, this study used raw scores for statistical analyses, because T scores are truncated (Achenbach, 2001). Standard scores and percentile ranks were reported only for informational purposes, to provide information on the functioning of the sample relative to norms. Background questionnaires were completed by both parents and adolescents at admission and discharge and assessed the adolescent’s treatment history, psychotropic medication use, legal Residential Outcomes., p. 5 record, grade point average, matriculation in school, communication with family members, compliance with rules, relationship quality, drug use, and alcohol use. In addition, residential program staff completed a brief form for each adolescent at discharge that indicated discharge status, length of stay, and problems that were the focus of treatment (type and number). Data gathered via the aforementioned measures were organized into the following groups of variables. I. Non-Treatment Variable Groups (1) Demographic Variables: age, gender, ethnicity, parental income, (2) Pre-Treatment Academic Functioning: grade point average, matriculation, (3) Prior Treatment History: psychiatric hospitalization, outpatient therapy, wilderness therapy, residential treatment, psychiatric medication, (4) Psychosocial Functioning: YSR Internalizing Scale, YSR Eternalizing Scale, YSR Total Problems Scale, CBCL Internalizing Scale, CBCL Eternalizing Scale, CBCL Total Problems Scale, Adolescent communication with family, Adolescent Compliance with Rules, Adolescent Family Relationships, (5) Presenting Problems: number of problems, mood disorder, substance abuse disorder, learning disorder, disruptive behavior disorders, eating disorders, anxiety disorders, developmental disorders, legal problems II. Treatment Variables Group: Adolescent Reaction to Residential Placement at Admission, Adolescent and Parental Satisfaction with Treatment, Length of Stay, Adolescent and Parental Report of Overall Improvement Over the Course of Treatment, Self-Reported Amount of Effort in Residential Treatment, and Discharge Status. The Discharge Status variable categorized students upon discharge, based on the clinical team’s recommendations. The five categories included: With Maximum Benefit (e.g. planned discharge, graduation from program, treatment goals met); Premature, With Program Approval (e.g. some progress, but student did not benefit from full treatment); Premature, Against Program Advice (program recommended continued stay due to clear need for additional treatment); Treatment Beyond Scope of Program (program not a suitable match, transferred to a more appropriate setting); and Other. The Treatment Beyond Scope category was included due to concerns that programs would be “penalized” for making appropriate treatment recommendations that included transferring students to a different level of care. In this case, it was deemed that a program making an early referral for students who required alternative clinical care would constitute appropriate, ethical care rather than a “failure” on the part of the program. Of the initial sample of 551 adolescents, 50 fell into the Treatment Beyond Scope category. A repeated measures ANCOVA was used to test the interaction hypothesis that those in the Treatment Beyond Scope category show less progress than those in the other groups. Analysis confirmed that for most of the YSR and CBCL scales, those in the Treatment Beyond Scope group did indeed show a statistically significant difference in their admit to discharge progress. Because of the foregoing the Treatment Beyond Scope group was excluded from subsequent analysis.